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SECTION  I 


INTRODUCTION 

The  concept  of  a well-posed  problem  was  formulated  by  Hadamard 
early  in  this  century.  In  broad  terms,  a problem  is  well-posed  in 
the  sense  of  Hadamard  if  it  has  a unique  solution  which  depends  con- 
tinuously on  the  data  of  the  problem.  Specifically,  if  T is  a trans 
formation  from  a metric  space  X into  a metric  space  Y,  then  the  prob 
lem 


Tx  - b (1) 

is  said  to  be  well-posed  if 

( i)  for  each  beY  there  is  a solution  xeX, 

( ii)  the  solution  x is  unique,  and 

(iii)  the  solution  x depends  continuously  on  the  "data"  b. 

A problem  which  is  not  well-posed  is  called  "ill-posed."  Ill- 
posed  problems  have  been  intensively  studied  during  the  last  fifteen 
years,  especially  by  Soviet  mathematicians  (see  [12], [23]),  because 
of  their  importance  in  many  engineering  applications  (see  [11]  and 
[ 14 ] for  specific  areas  of  Air  Force  interest).  In  this  report  we 
will  be  concerned  with  linear  ill-posed  problems,  that  is,  we  will 
study  the  problem  (1)  where  T is  a linear  operator  on  Hilbert  space. 
A typical  problem  of  this  type  is  the  integral  equation  of  the  first 


kind 


J k(s , t)x(s)ds  - b(t)  (2) 

a 

2 

where  the  kernel  k Is  a member  of  L ( [n,d]x[a,d] ) (the  space  of 

Lebesgue  square  integrable  functions  on  the  rectangle  [a,d]x(a,d]) 

2 

and  bfL  [a,d]  (we  allow  a or  d to  be  infinite).  Such  equations  are 
notoriously  ill-posed.  For  example,  if  k(s,t)  ■ t + c,  then  (2)  can 
have  a solution  only  if  b is  a linear  function,  violating  (i).  If 
k(s,t)  - sin(s)  and  b(t)  - 2,  then  by  the  well-known  orthogonality 
relations. 


f k(s,t)(l  + sin(ms))ds  - b(t),  m «■  2,3,.. 


which  violates  (ii).  Far  more  serious  is  the  fact  that  (iii)  is 
violated  for  equations  of  type  (2).  Indeed,  by  the  Riemann-Lebesgue 
lemma,  for  arbitrary  A, 

1 


/k(s,t)  A sin  (mxs)ds  -+  0 as  ra 


and  hence  solutions  do  not  depend  continuously  on  the  data. 

Numerical  methods  for  analyzing  ill-posed  linear  problems  are 
particularly  important  because  a large  number  of  engineering  prob- 
lems have  the  form  (2).  Consider  for  example  the  one  dimensional 
heat  equation 


x;:_ iJ 


2 


• fy  . u(x,0)  - h(x) . 
3x 


It  is  well  known  that  the  temperature  distribution  f (x)  * u(x,T)  at 
some  time  T > 0 can  be  expressed  in  terms  of  the  initial  temperature 


distribution  h(.x)  by 


f(x)  - — — /'exp(-(x-T)2/(4T))h(T)dx. 
2/vT  ^ 


The  "inverse"  problem  of  determining  the  initial  temperature  distri- 
bution h(x),  given  the  distribution  f(x)  at  the  later  time,  is  of 
considerable  interest  and  is  an  ill-posed  problem  of  type  (2). 

Another  problem  of  type  (2)  is  the  numerical  differentiation 
problem.  The  nth  derivative  of  a given  function  b(t)  (with  b(0)  ■ 
b'(0)  • ...  - b^n  ^ (0)  - 0)  satisfies 


/ (n-1) ! 


(t-s)n  ^xtslds  » b(t). 


This  problem  has  been  studied  extensively  within  the  context  of  ill- 
posed  problems  by  Cullum  [3],  Franklin  [5]  and  others. 

Another  example  is  afforded  by  the  work  of  Lee  [13]  and  Pro- 
vencher  {18]  on  the  determination  of  the  molecular  weight  distribution 
of  a solute  from  centrifuge  data.  In  this  example  the  molecular 
weight  distribution  f(m)  satisfies 


./iV 


2 2 -Xmx 


f (m)  dm 


1 - e 


3 


I 


where  U is  a function  which  is  proportional  to  the  measured  con- 
centration gradient  and  X is  a constant  which  is  proportional  to  the 
square  root  of  the  rotor  speed. 

As  a final  example,  we  give  the  two  dimensional  integral  equa- 
tion 


“ 6 - fj  (x.y)  - f9(x,y) 

which  was  studied  by  Singh  and  Paul  [21]  and  concerns  the  pressure 
distribution  in  the  contact  of  nonconforming  elastic  bodies. 

Integral  equations  of  the  first  kind  also  arise  in  the  deter- 
mination of  the  shape  of  conducting  bodies  from  backseat tered  elec- 
tromagnetic radiation  ([16], [17]),  seismic  prospecting  [2],  antenna 
theory  [4],  remote  probing  of  the  atmosphere  ([221, [24]),  medical 
tomography  [6]  and  system  identification  ([1],[15]). 


f E.Cx  \y.  ? 

J r / .. .»  \ 2 . 


,[  (x— x "* ) 2 + (y-y')"] 


SECTION  II 

GENERALIZED  INVERSES 

We  will  henceforth  assume  that  Hj  and  are  Hilbert  spaces  and 
that  T:H^  Is  a bounded  linear  operator.  The  Inner  product  and 

norm  in  each  space  will  be  denoted  by  (*,•)  and  ||»||,  respectively. 
The  range  and  nullspace  of  T will  be  denoted  by  R(T)  and  N(T),  re- 
spectively. Our  task  is  to  solve  the  ill-posed  problem  (1)  for 
xcH^  given  beH2*  Of  course,  if  b/R(T)  then  (i)  is  violated  and  there 
is  no  solution.  In  such  a case  we  might  reasonably  adopt  the  more 
flexible  attitude  of  replacing  b in  the  right  hand  side  of  (1)  by 
the  point  in  R(T)  which  is  nearest  to  b.  However,  if  R(T)  is  not 
closed,  such  a closest  point  may  not  exist.  We  are  then  led  to  accept 
as  a generalized  solution  any  vector  ueH^  which  satisfies 

Tu  - Pb  13) 

where  P is  the  projection  of  Hj  onto  RIT),  the  closure  of  R(T).  Any 
vector  u satisfying  (3)  is  called  a least  squares  solution  of  equa- 
tion (1).  We  note  that  a least  squares  solution  will  exist  for  any 

vector  b whose  projection  onto  R(T)  lies  in  R(T) , i.e.,  for  all  vec- 

x 

tors  b in  the  dense  subspace  R(T)  0 R(T)  of  H2*  It  is  not  diffi- 
cult to  see  that  least  squares  solutions  may  also  be  characterized 
as  vectors  ueH^  which  satisfy  either  of  the  conditions 

| | Tvi  — t»  1 1 <_  | |Tx  — b | | , for  all  x c Hj,  (4) 

or  * * 

T Tu  - T b 15) 

5 


where  T Is  the  adjoint  of  T (see  (7]  for  a proof  of  this  and  other 
simple  facts  pertaining  t this  section). 

We  have  seen  that  if  we  consider  least  squares  solutions  in- 
stead of  traditional  solutions,  then  difficulty  (i)  is  to  a certain 
extent  obviated.  The  problem  of  nonuniqueness,  however,  remains  at 
this  point.  Indeed,  if  N(T)  i (0)  then  there  may  be  infinitely  many 
least  squares  solutions,  for  if  u is  a least  squares  solution,  then 
so  is  u+v  for  any  vtN(T).  Fortunately,  there  is  a natural  way  of  se- 
lecting a least  squares  solution  which  is  unique  in  a certain  sense. 

We  see  from  (5)  that  the  set  of  all  least  squares  solutions  is  a 
closed  convex  set.  This  set  therefore  contains  a unique  vector  of 
smallest  norm  and  it  is  this  vector  which  we  will  accept  as  the 
unique  generalized  solution  of  equation  (1).  Let  $(T+)  - R(T)  © R(T)^! 


The  operator 


T^T*)  - 

which  associates  with  each  b e $(tS  the  unique  least  squares  sol- 
ution of  equation  (1)  with  minimal  norm  is  called  the  generalized 
inverse  of  T.  It  is  not  difficult  to  show  that  T is  a closed  linear 
operator  (see  [7]).  If  T^  were  continuous  then  problems  (i),  (ii), 
(iii)  would  be  solved,  at  least  for  bt:j5(T^  ) . But  alas  this  is  not 
the  case.  It  is  not  difficult  to  show  that  T^  is  continuous  if  and 


only  if  R(T)  is  closed.  Unfortunately  the  range  of  an  integral  op- 
erator is  closed  if  and  only  if  its  kernel  is  degenerate  (see  17]). 
We  are  therefore  led  to  seek  approximations  to  T by  bounded  linear 


H'PW'M  1 11  • - -"J'l 


operators.  Such  approximations,  when  applied  to  b,  are  called  reg- 
ularizors  of  equation  (.1). 


SECTION  III 
A GENERAL  METHOD 

* * 

We  will  denote  the  operator  T T by  T and  the  operator  TT  by 

T.  Note  that  T and  T are  self  adjoint  linear  operators  whose  spec- 

2 

tra  lie  in  the  interval  [0,||t||  ].  If  0 i o(T)  (the  spectrum  of 

t ,-l  * 

T),  then  by  (5)  we  have  T - T T . In  general,  however,  0 c o(T), 
but  this  last  equation  nevertheless  leads  us  to  seek  approximations 

t ~ * 

to  T by  operators  of  the  form  U(T)T  where  U is  a continuous  func- 

2 —1 
tion  on  [0,||t||  ] which  approximates  the  function  f(t)  = t in 

some  sense.  Specifically,  we  will  consider  a family  (net)  of  real 

valued  functions  {U„ (t) : BeS } , indexed  by  a subset  S of  the  positive 

p 

2 

real  numbers  with  ®cS,  where  each  is  continuous  on  [ 0 , ) | T J ) ] 

and  such  that 


|tUg(t)|  <_  M for  all  t and  (3 


(6) 


and 


uB(t)  -*  t 


-1 


as  M “ for  each  t + 0.  (7) 


The  following  is  proved  in  [7]. 

Theorem  1.  Suppose  bejtf(T^)  and  let  x„  ■ U„(T)T  b.  Then  x„  -*■  Tb 
p p p 

as  6 + “• 

To  this  we  now  add, 

Theorem  2.  If  bf?JJ(T+),  then  {x„}  h>\s  no  weakly  convergent  subnet. 

P 

Proof.  Suppose  (x  „}  is  a sublet  of  {x„}  which  converges  weakly 

p p 

to  zcH. , denoted  x ^ z.  By  the  v«ak  continuity  of  bounded  linear 

1 p 

operators  we  then  have  Tx  , ^ Tz. 


9 


ffUCCKUNO  FiOB 


. 


Pb  - Tx  - Pb  - TU  (T)T  b 
P P 


Pb  - TU^(T)Pb. 


However,  by  (6)  and  (7),  the  operator  TU  (T)  converges  polntwise  to 

p 

the  projection  of  H^,  onto  N(T)  * N(T  ) - R(T)\  Therefore 

Pb  - Tx,.„  -*  0.  It  then  follows  that  Pb  * Tz,  a contradiction.// 

p 

- * 

In  the  proof  above  we  have  used  the  fact  that  U,.(T)T  * 

p 

A > 

T U.(,T).  This  is  easy  to  see  if  U0  is  a polynomial.  In  the  general 

p p 

case  the  identity  follows  from  the  Weierstrass  approximation  theorem. 
Using  the  fact  that  bounded  sets  in  Hilbert  space  are  weakly  com- 
pact, we  have: 

Corollary  3.  If  b^<U(T^ ) , then  ||x^||  -*  <*>  as  B * «■>. 

Theorem  1 and  Corollary  3 demonstrate  dramatically  the 
unequivocal  nature  of  the  approximations  ix„}. 

p 

Several  authors  have  established  rates  of  convergence  for 
various  approximations  to  T*b  under  the  stronger  assumption  that 
Pb  » R(T)  fsee  l 20 ] , [ 9 ] , [ 101 ) . We  see  from  Corollary  3 that  the 
very  least  we  must  require  to  get  convergence  at  all  is  that 
Pb  t RtT).  In  order  to  strengthen  this  condition  only  slightly  and 
thereby  obtain  a rate  of  convergence  we  note  that 


R(T)  - R(TP  ) 
N(T) 


and,  in  the  polntwise  sense. 


10 


It  therefore  seems  reasonable  to  replace  the  hypothesis  b c$(T*), 
t.e.,  Pb  c R(T),  by  the  hypothesis  Pb  r R(TTV)  for  some  v > 0.  In 
order  to  gauge  the  rate  of  convergence  we  will  replace  (7)  by  the 
stronger  condition 

t V 1 1 - til  (t ) | < u>(B,v)  for  v > 0 (8) 

where  rnlR.v)  + 0 as  8 -*■  « for  each  v > 0 (the  case  v * 1 was  con- 
sidered in  [81). 

'V  x 

Lemma  4.  If  v > 0,  then  R(T  ) c:  NIT)  . 

Proof.  Suppose  (f^)  i*  a sequence  of  continuous  real  valued  func- 
tions on  [0,  | | T | | 2 1 such  that  f n ( t ) -*  tV  1 for  t i 0 and  tf^ft) 
uniformly  bounded  (for  example,  we  may  take  f^(t)  • tV  * for  t 1/n 

2fc\| 

and  f (t)  - n t for  0 < t < 1/n).  Let  {E. ) be  the  resolution  of 
n _ — t 

the  identity  generated  by  the  self-adjoint  operator  T.  Bv  the 
bounded  convergence  theorem  we  then  have 

,v  Jbll2  2 J ITI  I2 

Ty  - / t dE  y • / lim  tf  (t)dE  y 

0 0 n n 1 

J|T||2  . . i. 

- lim  / tfn(t)dEty  - lim  Tfn(T)y  c N(T)  .# 

n 0 n 

We  now  state  a rate  of  convergence  result.  The  vector 
T*b  will  be  denoted  by  x and  the  error  x - xrt  by  e . 


Theorem  5.  If  Pb  ■ TTVw,  where  v > 0,  then  ||e„||  < w(p,v) | |w ) | . 

P 

Proof.  Since  Tx  ■ Pb  - TTVw  and  since  x - t'w  c N(T)  , we  see 

that  x ■ TVw.  Now, 


xp  - Ug(T)T*b  - Up (T)T*Pb 

« « * t kX  1 

- Utt(T)Tx  - UfT)r w. 
P P 


Therefore  e ■ x - x - T (I  - U (T)t)w. 

P P P 

By  the  Spectral  Mapping  Theorem  and  Radius  Formula , we  then  have 


l«?p|  I < u)(p,v)  | | w | | . it 


In  our  next  result  we  become  more  cavalier  in  our 
assumptions  on  the  data. 

Lemma  b.  If  Pb  - f'w  where  v >_!,  then  ||ep||  i u>(S,v-l)  | | T o ^ | | ||w||. 


Proof . 


A * | 

As  in  the  previous  proof  we  find  that  x ■ T T w.  Also, 


« IL(T)T  Pb  - U (T)T  T w 
P P P 

- T*U0(T)fVw. 

P 

Therefore  cu  - x - x.  - T*( I - U0(T)T)TV'lw,  and 

P P P 

I |ep| |2  - (ep,T*(I  - U6(T)T)Tv‘1w> 

- (Tep,(I  - U^(T)T)TV_1w)  < «(P,w-l) | |w| | ||Tep|( 
Theorem  7.  If  Pb  - TVw  where  v > l,  then  ||e  ||^  < io(P , v)w(fl, v- I ) | | w | | . 


Proof . 


In  Lemma  6 we  s.iw  that 


0^  - T* (I  - Up(t)f)fv“lw. 


12 


therefore 


Te  - T*TV(I  - U (T)T)w. 

P P 

We  then  have 

||Te0||2  - (TeB.e6)  - (TV(I  - Ug(T)TV,Teg) 

< w(h,v)  | | Teg|  | , i.e.,  ||Teg||  <_ui(6,v). 

Substituting  Into  the  result  of  Lemma  6 completes  the  proof.# 

In  the  next  section  we  will  give  a number  of  examples  of 

specific  computational  techniques  to  which  the  above  results  apply. 

We  have  avoided  for  long  enough  the  problem  of  polluted 

data.  We  now  take  up  this  question.  Suppose  that  the  data  b Is  the 

result  of  measurements  so  that  Instead  of  b we  have  In  our  possession 

a corrupted  version  b'  satisfying  | |b  - bC | | <_  e.  We  operate  on  the 

vector  bE  to  obtain  the  approximations  xE  given  by 

p 

xj  - Ug(T)T*be. 

2 

Let  (>16)  - sup{ | tU-(t) | :t  e [0,||t||  ]},  and  recall  that  >16)  la 

p 

bounded  (by  (6)). 

Lemma  8.  | |Txg  - Txg|  | e $(P). 

Proof.  T(xq  - x*)  - TUfl(T)T*(b  - bC),  therefore 

P P P 

I lTxe  " ^e1'2  " (*(xb  " xp),xe  " XP 

- (TUg(t)T*(b  - bE),xg  - xE) 

- (tUp(T)(b  - bC),T(xg  - xg)) 
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j 


< #(6)116  - be||  ||T(xb  - xj)|| 


< E*(6)||Txp  - Txjl |.# 


Suppose  now  that  g(6)  ■ sup(|U  (t)|:t  e [0,||T||  ]).  Wo 

6 


note  that 


g(B)  -*•  ® as  8 -*■  •». 


Indeed,  If  this  were  not  the  case,  then  there  would  be  a constant  L 


such  that  III  (t)  | < L for  all  t and  6.  But  then  |tl)0(t)|  < Lt  » 0 

P P 


as  t > 0,  contradicting  (7). 


Lenina  9.  | |xp  - x^||  <_  tVg(8)#(6) . 


Proof.  Since  x0  - - T Ua(T)lb  - b ),  we  have,  bv  use  of 

6 6 6 


Lemma  8 , 


l|xp  - xjjll2  - (xe  - Xg,T*U^(T ) (b  - bE)) 


(T(Xp  - xp) *Up(T) (b  - br)) 


< e #(8)g(8).# 


Suppose  now  that  Pb  - Tw  (wo  could  also  use  the  other 


hypotheses  considered  above,  but  we  choose  to  consider  this  simple 


case  to  illustrate  the  Ideas).  By  the  triangle  inequality  we  have 


I*  “ x[||  | £ | |x  - xp||  + 


1*8  " V 


Lemma  9 and  Theorem  7,  then  give 


Theorem  10.  jf  Pb  - tw,  then 


1 


SECTION  IV 


SPECIFIC  METHODS 

In  this  section  we  will  consider  some  specific  choices  for  the 

functions  {U0(t)}  and  we  will  find  functions  u(0,v)  which  determine 
P 

rates  of  convergence.  The  index  set  S in  all  examples  below  will  be 
either  the  set  of  nonnegative  reals  or  nonnegative  Integers.  In  the 
discrete  case,  the  parameter  £ will  be  denoted  by  n. 

As  a first  example  we  consider  Showalter's  integral  formula  [19]: 

°o  ^ 

T b - / exp(-uT)T  bdu. 

0 

The  functions  U.  for  this  example  have  the  form 
P 

6 

Ue(t)  - / exp(-ut)du 
3 0 

and  may  be  motivated  in  terms  of  Borel  summabillty  [7].  It  is  not 
difficult  to  see  that  a function  w(B,v)  satisfying  (8)  is  given  by 

uHB.v)  - B V (v  > 0). 

The  choice  UQ(t)  ■ (t  + B *)""*  (8  > 0)  leads  to  Tychonov's  reg- 
P 

ularlzatlon  of  order  zero  [23].  Here  one  can  readily  verify  that 

b>(Btv)  ■ B V for  0 < v <_  1. 

In  order  to  obtain  approximations  with  this  rate  for  v > 1 we  may 
use  extrapolated  regularization  [9].  That  is,  for  a given  B > 0 we 
set 
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' 


U<0)(t)  - (t  + 6 V1 
and  define  Richardson  extrapolants  by 

U^j)(t)  - (2jU^-1\t)  - U^_1)(t))/(2j  - 1), 

j = 1,2,  • . . • 

It  is  not  difficult  to  show  (see  [9,  lemma  2.1])  that  for  k = 0,1,2,... 

tk+1|l  - tUgk)  (t)  | = 7T  (-j-^ — ) 

P i=0  2xRt  + 1 

< a-""1 

Therefore,  for  the  kth  extrapolant  we  may  apply  Theorem  7 with 

-k-1  -k+k 

u>(6,k)  = 8 , k = 1,2,...,  to  obtain  the  rate  8 (see  [9,  The- 

orem 3.2]). 

We  now  consider  some  iterative  regularization  methods.  Below, 

_2 

a will  be  a parameter  satisfying  0 < a < 2 ||t|| 

If  the  functions  11^(0,  n = 0,1,2,...  are  defined  by 

n , 

U (t)  = a X (1  - at)K 
k=0 

then  (6)  and  (7)  are  satisfied  and  one  can  show  that 

nV|l  - tUn(t)|  = nV|l  - «t|n+1 

is  uniformly  bounded.  From  this  we  find  that  the  rate  of  conver- 
gence of  the  iterative  process 

* , - * 
x = aT  b,  x . = (I  - aT)x  + aT  b 
o n+1  n 


18 


£ 


One  can  verify,  as  in  the  first  iterative  example  above,  that  the 
function  ui(n,v)  ■ n V determines  a rate  of  convergence. 

The  iterative  method 


Xo  = T*b>  xn+l  = Xn  + (T*b  " txn)/(n  + 2)’ 

was  investigated  in  [10].  The  appropriate  functions  are  given 
by 

n . k-1 

U it)  - £ (k  + l)"1  ']J  (1  - t/(l  + j)). 

n k-0  j=0 


This  leads  to  the  iterative  method 


xo-T*b.  x„+l  ‘ xa  + (T  b - i'Xn>,<"+2) 


Following  the  analysis  given  in  [10]  one  can  show  that  the  rate  of 
convergence  of  this  method  is  governed  by  the  function  u>(n,v)  = 
Uog  n)  V. 
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